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[57] ABSTRACT 

A system for enhancing the quality of an image that 
suffers from staircase effects, contouring effects, and 
granular noise, without affecting the existing method of 
creating the image is disclosed. The enhancement sys- 
tem identifies the edge and nonedge regions in the de- 
coded image, and then filters those regions in order to 
reduce the specific type of quality degradation that 
generally occurs in the specific type of region. The 
system is integrated into a processing system including 
means for receiving and displaying digital image data. 
The system receives the digital image data, differenti- 
ates between edge data and the nonedge data within the 
digital image data, and filters the image data accord- 
ingly. The filtered edge data and the filtered nonedge 
data are combined to form an enhanced image. A multi- 
pass filtering system is also disclosed. 

11 Claims, 7 Drawing Sheets 
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_.„,.^ one dimension of the pixel can be described. Vector 

IMAGE ENHANCEMENT SYSTEM encoding schemes are useful for image coding when the 

•rc/-'iixTi/^A» av.^. „^ image is made up of elements having both color and 

TECHNICAL AREA OF THE INVENTION shading characteristics, Le.. intensity as well as for the 
This invention relates to a system for image enhance- ^ coding of other analog data or very high rate discrete 

ment and, more particularly, to a system for enhancing example, an image may be comprised of pixels 

the quality of a decoded quantized image. having 8 colors, each of which can assume 4 shades, 

n A nvnp riTTxiT-i rtx? T-tTT- iKT* rr-xT^^^t V^^^^ valucs could then be represented by a 

BACKGROUND OF THE INVENTION 2-dimensionil vector. The first vector field is a S-bit 

As the state of the art of digital signal technology binary value for color; the second vector field is a 2-bit 

advances, related technologies such as digital image binary value for shade. Thus, in order to ewctly encode 

processing experience corresponding growth and bene- the image, each vector would require 5-bits of data to 

fits. For example, the development and proliferation of represent a single pixel value. 

facsimile communications allows images to be encoded To reduce the size of the encoded digital image, bit- 

mto distal signals, transmitted over conventional tele- to-pixel ratios other than one-to-one or many-to^ne are 

phone line and decoded into a close representation of . used. A lower bit rate reduces the amount of digital 

the ongmal unages. Image data are also digitally en- image data that must be stored, transmitted, etc. A num-^ 

coded for ease of storage, modification, copying, etc. ber of quantization schemes have been designed to con- 

iil 1??!^*?" ♦'^^''"''^ growing technologies. trol data encoding and decoding in order to produce a 
the field of digital unage processmg is also experiencing ^ lower bit rate 

'tS'^'Sr^^fSi^^^^ ""r' quantizers map . of unique input 

ina\ rAiaf*. »-.«o«iur*^ ? • u i^^i^^ ^ vcctors onto a smaller number of output vectors in 

In o?der to infrease the eff^ency ml tSoTe X 25 ^^^J^^^ "^P"^ TkT.°"^^vL*^ r^^' !5 

usefubiess of digital image decoding schemes, the cod' T^"' detennmed by a quan- 

ing system must provide a coded seT of image data that Ti^T'^*^ ? berepresentatrve of the mput vector 

is more efTicient to store, transmit, etc., than the original ^^^^^^^ '^""^ ^^"^ "^P"^ 

image data and must reproduce a decoded image with '^T'f*' * "'"^'^ represent all of 

some minimum level of quality. However, the conver- 30 ^^J^" vector set requires only a 5-bit 

sion of relatively high bit rate image data to lower bit "^P"^ ^ mapped to the output vector 

rate data virtually always entails a loss of image quality. "closed" to the input vector. For example, if the 

One straightforward method for coding an image is "^P"* ^^'^ °"^P"^ vectors are (3, 

to create an artificial grid over the image aiid to assign ^P^^ vector would be mapped to the 

a value to each grid space representing the color of the 35 o"<P"^ vector (3. 4) because it most closely represents 

original image at that grid space. If the grids are made "^P"* vector. The goal of the mapping is to obtain a 

small enough and the values represent a large enough cluster of input vectors around each output vector, 

range of color, then the image may be encoded and "*P^^ vector is then represented by the closest 

decoded with small image quality degradation. For output vector. The output vectors are then encoded 

example, display screen images are made up of an array 40 ^ binary code. When the output vectors are de- 

pf pixels, i.e., picture elements. On a black and white coded and the decoded image assembled, the decoded 

screen, each pixel has a value of one or zero represent- image lacks the detail of the original image. The de- 

ing the on/off state of the pixel. In a one-to-one bit-to- coded image is constructed only from the output vec- 

pixel coding scheme, each pixel value is represented as ^^rs, which do not correspond exactly to the original 

a 1 or as a 0 and the entire screen image is encoded. The 45 iiiput vectors. One of the major focuses of vector quan- 

r^ult of the encoding is an array of binary values. To tization design is the balance between the size of the 

decode the image, the array values arc translated into a output vector set and the bit rate, 

screen image having pixels on or offin the same order in Decoding images, constructed by a low bit rate 

which they were originally encoded. scheme, generally suffer from the following types of 

If the image is comprised of more than two distinct 50 degradations; (a) edge regions in the decoded image 
colors, then more than a 1-bit code must be used to contain staircase effects, (b) quasi-constant or slowly 
represent the pixel values. For example, if four distinct varymg regions suffer from contouring e£fecU. and (c) 
colors are present in the image, a 2-bit binary code can textured regions lose detail and suffer from granular 
represent all of the values. If the image includes 256 noise. These problems arise in images encoded/decoded 
distinct colors, a 8-bit binary codes is required to 55 by both scalar and vector quantization schemes. For 
uniquely represent each of the color values. The mem- low bit rate coding applications, an acceptable rate of 
ory requirements for such coding schemes increase as transmission lies in the range of 0.2 bit/pixel to 1 bit/- 
thc number of distinct colors in the image increases. pixel. At these rates, the staircase and contouring effects 
However, with high bit rate coding schemes, the qual- are pronounced and affect the visual quality of the de- 
ity of ^e image that results from the decoding of the 60 coded image. 

digital data will be good as long as the digital image Staircase and contouring effects in a decoded image 

transmission or recovery from storage is successful. are generally fairly obvious to the naked eye. The stair- 

When an encoding scheme converts a single value, case eiffect is the blockiness that occurs at an edge re- 

i.e., a single value representing a pixel, into a coded gion, Le., a region wherein two distinct pixel value 

value, the scheme is commonly referred to as scalar 65 regions meet. A staircase effect is produced by the fact 

encoding. A vector encoding scheme is similar to a that there are fewer output vector values than input 

scalar scheme except that input vector data rather than vector values and thus the intensity changes along de- 

scalar data are encoded. With vector data, more than coded edge regions are more pronounced. Additionally. 
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most quantizers tend to convert diagonal edge regions formed according to whether the image data is identi- 

mto staircase type regions due to the encoding schemes. fied as edge or nonedge data. 

The contouring effects that appear in the slowly vary- In accordance with other aspects of the present in- 

mg regions are also be caused by the fact that not all of vention, prior to differentiating between the edge and 

the variations in the intensity of the original image are 5 nonedge data, a subimage of the digital image is formed, 

available for the decoded image. For example, if a re- The edge and nonedge data comprising the subimage 

gion of the original image included an area having 4 are differentiated and filtered. The boundaries of the 

intensity changes therein, the decoded image might image are not filtered because, of the sliding window 

represent the area with only 2 intensities. In contrast to filtering technique that is used. The method includes a 

the staircase effects and the contouring effects, the ef- ^0 final step of integrating the nonfiltered portion of the 

feet of the granular noise on the viewed image is often digital image with the enhanced subimage to form a 

mitigated by the very nature of the textured regions. complete enhanced image. 

However, some blockiness is still likely to be visible in J° accordance with additional aspects of the present 

these regions due to the lower bit rate, i.e., the decrease invention, the step of differentiating between edge and 

in intensity variation. IS nonedge regions is carried out by analyzing the digital 

Each of the degradations represents a loss of fidelity image via an edge detector. Further, the edge filtering 

of the decoded image from the original image. In vector means is a median filtering process and the nonedge 

quantization systems, the three common types of degra- filtering means is a statisical D-filtering process. 

^^^l^r,!^- f 20 BWEF DESCRIFnON OF THE DRAWINGS 

mg values, the distortion measure used m the quantizer ^ 

design, i.e., how close the input values must be to the * ^^^^ diagram of the enhancement system 

encoded values, and the finite training procedure in- present invention; 

volved in the design of the quantizer. Most present sample decoded digital image {D(m,n)}; 

systems for enhancing decoded images are directed ^ ^ ^ diagram of the edge detection pro- 

toward modifying the operations of the quantizer or <^of the present invention; 

encoding system and/or modifying the bit rate. ^' * ^ ^ diagram of the filtering process of the 

The present invention is directed to a system for ^^^!!} ^yention; 
enhancing the quality of a decoded image at the image . . ^ " example of a filtenng window for filter- 
decoder without affecting the operation of the quantizer ^ miage region via the process carried out in accor- 

or other encoding method and without increasing the ^° ^^^^ ^""^ t- 

bit rate. Because the system does not affect the encod- ^ ^^"^ P[ a multi-pass enhance- 

ing or transmission processes, it can be readily inte- "^^nt system in accordance with the present myention; 
grated into established quantizer systems. The present • ^^^'Z ^ ^""Z "^"""^ ^^i^™^ 

invention is suitable for^nhancing any decoded iS l^^^lT' "^^^^JS^V^^*^'" accordance with 

whetheritisastoreddigitalimageor^imagereceivfd'' "^l^r « lT>^f h . • 

in real time. » © e pjQ ^ ^^^^ diagram of a vector quantization 

and digital image transmitting system into which the 

SUMMARY OF THE INVENTION "nage enhancement system is integrated. 

The present invention provides a system for enhanc- 4q DETAILED DESCRIPTION OF THE 

ing the quality of an image that suffers from staircase PREFERRED EMBODIMENT 

effects, contouring effects, and granular noise, without tvit>i r^fpr^nn* tr^ jsm i ' u 

ve^rquanuzauon process or the bit fate of tran«nis- 50 fflteriBg process. The one^KjIe-two-way fUter switch^ 

invtr^r^ with additional aspects of the present {J>(rn)fto°Ki«^ftrS^r « T^^^^^^ 

mvention. the system 1$ mtegrated into a processing the appropriate filtir output to an output which consti- 

system mcludmg m««s for r«:e.ving and displaying an tutes L enh«.ced im^e {ECm.n)" TT,e «hS 

mage compr»ed of digital data. TTie method of the 55 image {E(m.n)} has impfovedim^e qudity we7^ 

present mvention comprises the steps of receiving a ^age {D(m,n)} 

^^^^^^ «>8e ^ "4 the The decoded image degradation is minimized by 

nonedge data withm the digital image data, and fUteriiig filtering the nonedge portions of the image with a filter 

the image data accordingly so as to decrease the stair- that smooths the blockiness. contouring: and noise ef- 

case effecte appeamig m the edge date and to decrease 60 fects of the decoded image. The filtering process inte- 

the loss of detail Md contourmg effecte qipearing in the grates some of the pixel value variation that was de- 

nonedge data. The method also includes the step of creased during the coding process back into the de- 

mtegratuig the filtered edge data and the fUtered no- coded image. The edge portions of the image are 

nedge date mto an enhanced image. smoothed by the edge filter in order to decrease the 

In accordance with still further aspects of the present 65 staircase effect at the edge regions. 

BiVHition, the method includes a second filtering step to Control of the enhancement quality is obtained by 

fiirther decrease the nnage degradation in the enhanced adjusting the values of the edge detector parameters n 

image. The filtenng of the image date is again per- and the filter parameters N,. The exact parameteis and 
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their effect on the image enhancement process depend hanced by the enhancement system because of the posi- 

on the specific edge detector and filters that are used. tioning of the filter windows in statistical D-filtering 

Sr^ tw nij?"^?^""' ^'^f P^^"^"- median filtering techniques. At the end of the en. 

^L^L S^^t ^"T^Vlu" ^^T!?' *° hancement procedure, the unfiltered decoded image 

J^hT^pW^K^ • "^^^^^ ' ^^^^ is combined with the filtered subimage 

tcrs that define the size of the unage area to be filtered. fjs^^^w a *v » * & 

In one actual embodiment of the present invention, an SS^tf ^ ^Z T .^T^^*^''^^f 

edge detector referred to as a Kundu-Mitra edge dclec- J^*^ ^^^^^ f « >s chosen in relation to the 

tor. a statistical D-filter, and a median filter are imple- ^"^^^"^ P^^"* example, the bor- 

mentcd as the edge detector 12. nonedge filter 14 and lo '^^L'.'^f Pf , ^ . . 

edge filter 16, respectively. The Kundu-Mitra edge reference to FIG. 3. the edge detector analyzes 

detector is described in A. Kundu and S. K. Mitra, **A subimage {Ds(m,n)} and generates a corresponding 

New Algorithm for Edge Image Extraction Using Sta- subimage {Bs(m.n)}. The edge subimage is a bi- 

tistical Classifier Approach," IEEE Transactions on '""^ image with each pixel Bs(m,n) taking on a value 

Pattern Analysis and Machine Intelligence, July 1987, pp. 13 ^on^ set {0, 255}. The edge detector treaU pixels as 

569-477. A statistical D-filter is described in J. L. having one of only two values, e.g., low or high. The 

Hodges, Jr. and E. L. Lehmann, "Estimation of Loca- edge detector detects an edge if the pixel value is low 

tion Based on Rank Tests," Annab of Mathematical and the pixel is in a two-value region. If the edge detec- 

Statistics, 1965, pp. 598-611. A median filter is described tor sets B5(m,n)=0, then D(m,n), the corresponding 

in T. A. Nodes and N. C. Gallagher, Jr., *'A generahza- 20 P«e} in the decoded image, is a pixel belonging to an 

toon of Median Filtering using Linear Combinations of edge. On the other hand, if the edge detector sets 

Order Statistics," IEEE Transactions Acoustics, Speech, B5(m,n)=255, then D(m.n) is a pixel that does not be- 

and Signal Processing. Vol, ASSP^30, October 1982. pp. long to an edge, i.e., is a nonedge pixel. 

. ^ ^ ^ . . With reference to FIG. 3, at block 20, the subimage 

ot^Jin^^ nn;^^^^^ '^f '^^u'^'V coordinates are initialized to the lower left comer pixel, 

operation is controlled by the values assigned to thresh- ; ^ i« r^^^ u^j- * j j 

old parameters n, t2. and 73. Empirical studies indicate e.;nTLcr.t '^^^^^^ k^' 

that suitable value ranges for theie parameters are: f^J f °^ the submiage 

*^ {D5(m.n)} and scans the subimage from left-to-nght, 

20Sti S40; bottom-to-top. At block 22, a sequence {a/}, for i =0 to 

30 8, is generated. The sequence {a,} is defined such that: 

8^72^20; 

A higher value in the range for the threshold parameter 35 a,- = Z>5 ^^m - i + f-^l, (n - 1 + i mod 3) \ 
T\ causes the edge detector to detect stronger edges, i.e., 

edges that are composed of at least two values that are t?^ 1 r 1 • 1 . t. • * ^ 

considerably different thereby producing a sharp edge ""^ ^ ^ ^"^f^ "*'T' ""l 

SimUarly, a lower value in the range fof the toeshold ''^y""?^ ^ the values for a 3x3 mage 

parameter T2 results in the detection of stronger edges. 40 !'*'fl°P centered around Ds(m,n). The vanable 04 

On the other hand, a higher value in the range for the * ffl'^^ ^ r -v . . . 

threshold parameter T3 results in the detection of thin- ^^^^ sequence {a/}, for all 1, is sorted to 

ner edges, i.e., detected edges are close to one puel Produce a new sequence {a(,)} such that: 
thickness. 

In the statistical D-filter and the median filter, the 45 ^ • • ^a(8,. 

window size parameters Ni and N2 are assigned values -ru ^ j *- 

to affect the filtering operations. Generally, the window condiuon, 
size parameters assume values from Z+={1, 3. 5, 7, . . 

. }. Desirable parameter values or value ranges have r -1 

been determined by empirical studies of viewer re- 50 (gk?) t a<i)) > ti and 04 < -^^LlfilL I 

sponse to a comparison of the original image and the ^ J 
enhanced decoded image and to general quality of the 

enhanced decoded image. In the present invention, the *s tested at block 26. The first part of the condition tests 

parameter values are controlled interactively or may be whether the high and low values in the region, approxi- 

initialized prior to starting the enhancement process. 55 "^^'^^V by OiJ) and a<i), respectively, have a difference 

With reference to FIG. 2, the decoded image is de- ff'^ter than the threshold parameter ti. The second 

noted by {D(m.n),l^m^M,l ^n^N}, where the inte- condition test is whether the value of D(m,n) 

ger pair (m^i) denotes the spatial coordinates of a pixel ^ ^ "low" value. By detecting only low values in two- 

and the image size is M XN pixels. The notation D(m,n) ^^^^ regions, the edge detector traces the contour of 

denotes the intensity value of the pixel. In the present 60 low pixel value at any edge in the image. Clearly, by 

example, each pixel has 256 levels of grey. Thus, reversing the comparison sign in the second part of the 

D(m,n) assumes integer values between 0 and 255. condition test, the edge detector would trace the con- 

Upon receiving the decoded image {D(m,n)}, a tour of the high pixel values in the inaage, 

subimage {Ds(m.n)} is identified within the decoded If the edge condition is TRUE, then the region is 

image. The subimage is defined as {D5(m,n)=D(m,n) 65 considered to possibly be a two- value region. At block 

for (a+ l)^m^(M-a) and (a+ l)^n^(N-a)}, 28, if the condition is FALSE, D5(m,n) is considered to 

wherein a is the border size parameter. A border that is be a part of a constant region, and B5(m.n) is set equal 

a pixels wide around the image {D(m,n)} is not en- to 255; that is, D5(m,n) is identified as a nonedge pixel. 
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At block 30. the edge condition tested at block 26 is determination. The filtering process produces filtered 

TRUE and the value at Ds(m.n) might be an edge. To values for {E(m,n)}. If a pixel is idenUfied as an edge 

determine whether Ds{m,n) is an edge pixel, the size of pixel, then the median filtering produces a value that is 

the region under consideration is increased and the edge equal to the middle-most value in the region. The steps 
detector attempts to detenmne whether the region is a 5 described in relation to blocks 5^-60 generally describe 

two-valued edge region or a nonedge region having the median filtering process. If a pUel is identified as a 

textured portions. A second sequence {0th for i=0 to nonedge pixel, then the statistical D-filtering causes the 

fined h^h ^ ' sequence /}, is dc enhanced pixel value to be set to an approximated mean 

^ , value for the region. The steps described in relation to 

/»/«i?5{(m-2+[.y5i). (n-2+iinad 3)) l**^^ ^^'^^ ^^"^ Statistical D-filter- 

mg process. ■ 

The sequence is made up of the values for a 5 X 5 region Beginning at block 52. the image coordinate parame- 
that is centered around Dsim^n), The sequence {ftjTfor agam set to m=n=4 so that the filtering process 

all i, is sorted at block 32 in order to produce a new i< ^^^^ ^^^^^'^ subimage 

sequence {fiiij} such that: lD5(m,n)>. Window position parameters are initialized 

as 

At block 34, a variable yis calculated as: j-^ ^ J^j j-^ ^ 

y = I A window position parameters T/ are used to define 

* ' the position of the filtered pixel region around the pixel 

. ... . „ D5(m,n). At block 54, the value of Bs(m,n) is tested to 

Additionally, a counter S is set to 0. At block 36, a set determine whether D5(m,n,) is an edge pixel, 

of conditions R,-, for all i. is calculated as At block 56, with B5(m,n)=0, a sequence {u/}. for 



i=0 to ((N|)2-l), is defmed such that: 



- Ti + [iTp]). (n - + I mod Ni) ^. 



The counter S is incremented for each condition in {R/} 

that is TRUE. The high value in the region is approxi- r i . , ^ ^ , ^ . , 

mated by ^(22). At block 38, a second edge condition, sequence {u/} mcludes the values for a region of 

35 lD{m,n)} that is defined by the window size parameter 
Ni and positioned according to position parameter Tj. 

j S~ 12.5 I With reference to FIG. 5, the filtering region for the 

I I ' pixel {D(4,4)} with window size parameter Ni=5 is 

shown. The 5X5 region has its left lower comer posi- 
is tested. The condition tests the binomial distribution of 40 tinned at {D(2,2)}. The filter windows for {D(4,6)} and 

the low values over the region compared to a probabil- {l>(6,4)} are shown in reference and illustrate the repo- 

ity limit represented by T3. If the condition is TRUE, sitioning of the filter window relative to the pixel being 

then the region is considered to be a two-value region. filtered. 

At block 40, B5(m,n) is set to 0; that is, D5(m,n) is an In the present example, a 3-pixel border is left unfil- 

cdge pixel. Otherwise, at block 28, B5(m,n) is set to 255; 45 tered. The size of the border allows the system to use a 

that is, I>5(m,n) is a nonedge pixel. The edge detector 7*7 maximum size filter window, i.e., Ni = 7. If a 7X7 

acts to mitigate some of the staircase effect by not iden- filter is used, the lower left comer of the filter wmdow 

tifying all of the blocky edge comers as edge pixels. for {D(4,4)) is positioned at (1,1), i.e., a comer of image 

Rather, the edge contours detected by the edge detec- {D(m.n)}. If the window size parameters are preset, the 

tor are somewhat smoothed. 50 border parameter may be set so that the maximum 

At block 42, the system determines whether the en- subimage is filtered. For example, if it is certain that Ni 

tire subimage has been scanned by the edge detector. If and N2^3 then the border may be narrowed to 1-pixel 

the condition [(m=m-a) AND (n=N-a)] is TRUE, size. In such a case, the comer of the filter window for 

then the last pixel that was considered, at coordinate the pixel at {D(2,2)}, one of the comer pixels in the 

(m,n), was the upper right hand pixel in the subimage 55 subimage. would be positioned at (1.1) and the filtering 

and subimage {B5(m,n)} is complete. Otherwise, at process would function properly. However, in order to 

block 44, the coordinates (m,n) are set to the next subim- allow the interactive selection of values for the window 

age pixel to the right, or, if the end of a row is reached, size parameters, the border parameter size must be ade- 

to the first pixel m the next row up. If m=M-a. then quate to accommodate the largest window filter size 

the end of the subimage row has been reached and m is 60 that might be selected during the enhancement process, 

reset to a + 1 and n is set to n-l- 1. Otherwise, m is set to Referring again to FIG. 4. at block 58, the sequence 

m + 1 . so that the next pixel to the right in the same row {u/}, for all i, is sorted to produce a new sequence {xirnh 

of the subimage is considered. such that: 

With reference to FIG. 4. the generation of the en- 
hanced image {E(m,n)} includes the steps of dctennin- 65 u(0)Sim)au(2)^ . , . gu((A^j)2_i). 
ing from the edge subimage {Bs(m,n)} whether a given 

pixel is an edge or nonedge pixel and then filtering a At block 60. the filtered image pixel E(m,n) is set to 

region of {D(m,n)} around the pixel according to that «((M)2-iy2, i.e., the middle value in the sequence. By 
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scmmng and filtering the decoded image in a uniform filter 64 and edge filter 66, and switches 68 and 69. The 
fashion and by consistently selecting the median value fUters 64 and 66 are nonedge and edge filters similar to 
as ine tutted value, the edges in the decoded image are fiUers 14 and 16. During the second pass of filtering, 
aS.S '^"^^'Lr . ■ ^"'^ *e window rize parameters Nj and N4 of 

al"il^'e;^^:lfS'^^^^^^^^^ ' filters 64 and 66. resp..vel:r .^ 

to ((N2)2- 1), is defined, such that- ^"^^ '""^^^ ^ ^'S^^'y different manner than was 

done by filters 14 and 16. For example, it has been found 
that parameter settings of NjNa^S or 7, N2=3 and 
f f • r I "l^ N 1ft N4=l minimize or essentiaUy remove the staircase ef- 

= ^.(^(^^ [l^j}in - 73 + imo<iN2) J- ^^^^ ^ ^^^^ quantizer. 

With reference to FIG. 7, the steps invovled in the 
At block 64. the sequence {y/}, for all i, is sorted to generation of {E2(m,n)} are similar to the steps for 
produce a new sequence {y(/)}, such that: generatmg the first enhanced image {EKnun)}, At 

y(0)=y(j)=y{2)^ . • - ^yum-i). 15 block 70, coordinates m and n are set to 4 for a 3-pixel 

. border around the subimage. Additionally, the window 
At block 66, a new sequence {zi^fon^O to position parameters are initialized as 



- I) 

2 20 

is defined, such that: 



^3 = [J^>n.[-L]. 



As block 72, if B(m,n)=0, that is D(m,n) is an edge 
m + WA5)2-i-o P"^'' *^en at block 74, a sequence {q/}, for i=0 to 
2 • ((N3)2-1),U defined, such that: 



The sequence {z/} includes the pair-wise averages of 

values from {yo)}^At block 68, the sequence {z/}. for all . ( ( « - ^3 + 1^1^ (n - n + / mod N,) \ 

1, IS sorted to produce a new sequence {z(o}, such that jq \n L J/ ) 

Jto) s g . . . s r {(Ai)J-i). sequence {q/}. for all i, is sorted at block 76 to 
generate a new sequence {q(o}» such that: 

At block 70, the filtered image pixel E(m,n) is set equal 35 

to Mitrm-w, i.e., the middle value of {z(o}. E(m,n) is the image pixel (m,n) is set to equal 

thus set to an approximated mean value of the region. <i((iV3)2-i)/2, i.e.. the middle value in the region. 
By scanning and filtering the decoded image in a uni- At blocks 80 and 82, the coordinates (m,n) are up- 
form manner, the nonedge region values will be dated to the next pixel m {E5'(ni.n)} in a manner similar 
Knooihed, i.e., the values across a nonedge region will 40 to that carried out at blocks 42 and 44, respectively, in 
be made more uniform. Additionally, because of the FIG, 3. 

method by which {z{,)} is created, the number of unique If B5(m,n)=:255, that is D(ni,n) is a nonedge pixel, 
pixel values in the image may be increased. In this man- then at block 84, a sequence {v/}, for i=0 to ((N^)^- 1), 
ner. the contouring effects that appear in quasi-constant is generated such that: 
or slowly varying regions and the noise effects in tex- 45 
tured regions will be reduced. 

At block 72, the system determines whether all (( f 1 1^ ^ 

subimage {D5(m,n)} values have been filtered. At block " - n + L"Ar J j' " ^* + ' o*** ^-*> I- 

74, if the subimage filtering is not complete, then the 

coordinates (m,n) are updated and the filtering process 50 tk* \ f« -n • - j * ui 1 . 

repeated. The steps at bloTks 72 and 74 are simSa^ to the W>. ^} \« at block 86 to 

steps described ii conjunction with blocks 42 and 44, P"^"^' * ^^«>* ^^at: 

req>ectively, of FIG. 3. , ^ ^ ^ ^ 

Tlie filtering of the complete subimage {Ds(m,n)} -<«)^^0)--a)^ - • - W-i). 

produces the enhanced subimage {E5(m,n)}. At block 55 a* kwit m . .^..•^x. /„. i j n 

75, the combination of the enhanced subimage with the ^* ^ "^"^"^^ ^"^^^ for i=0 to 
border of the decoded image {D(m,n)} produces a com- 
plete enhanced image {E(m,n)}. - i) 

Empirical studies indicate that a second filtering pass 2 * 

whereby the first enhanced subimage, hereafter referred 60 
to as {Ei(m,n)} is filtered to produce {E2(m,n)} further is defined, such that: 
enhances the quality of the output image. As an example 
of one preferred enhancement system for multi-pass 

filtering, with reference to FIG. 6, the system includes + nm^-x-t^ 

the edge detector 12, the filters 14 and 16, and the 65 , ^ 
switches 18 and 19 described in conjunction with the 

single-pass filtering system. Additionally, the enhance- The sequence {w;}, for all i, is sorted at block 90 to 
ment system includes a second set of filters, nonedge produce a new sequence {w(i)}, such that 
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enhanced images generated by systems using different 

w(0) ^ 1^1) ^ M^) g . . . £ w ((AE4)2-i) . parameter values may be produced and compared. The 

2~' enhanced image with the most satisfactory quality can 

then be chosen and used as the final enhanced image. 

At block 92, the image pixel E2(m,n) is set to 5 While preferred embodiments of the invention have 

W((M)2-1V4. Beginmng at block 80. the coordinates been illustrated and described, it will be appreciated 

^n) are updated to the next pixel in the subimage. that various changes can be made therein without de- 

fiiT^Vt? tli T"*^^ pixel from {E.i(m,n)} has been parting from the spirit and scope of the invention. Gen- 

fro^ /pw VV f set to the border erally, the invention is suitable for use with any number 

{E?m nV^ "^""^ "^^"^ of image processing systems that process digital images 

Empirical studies illustrate the various enhancement ^""l previously ^coded or process^^^ a 

qualiti« obtained for different win^w si^T^e e '^A^'^u 1 

settings. If N, =N2=N3=N4= 1. the enhancement sys- • '^'^^ *^ 

tem produces an enlJnced im;ge {EV^)} K 15 l^^^lSrV^^^" ^ ' ^ 

identical to the decoded image {D(m,n)}. i.e^an identi- ^^^^^^^ed Laplace operator and a thrcsholded Sobel 

cal unenhanced image. If Ni=3 and N2=N3Ni=l a oP^^^or may be used as the edge detector. MulU-pass 

slight improvement is notices on edge areas and disc<Jn- of the image may be done with various combi- 

tinuities present on the edges are moderately smoothed. ^^^""^ °^ and nonedge filters being used. 

If N|:=N3=3. then the edge smoothing is increased. If 20 ^ additional benefit of the present invention is that 

Ni=N3=5, and N2=3. moderate improvements are analyzing the decoded image {D(m,n)} via an edge 

perceptible in the edge and nonedge regions. If Nj and detector the edge information may be used for other 

N3 are increased to 7, additional improvements are visi- processes. For example, it has been found that an addi- 

ble in both edge and nonedge regions. Finally, if N2> 3, tional reduction in contouring effects in the nonedge 

the nonedge regions are smoothed to a great extent and 25 region is obtained by adding tmiformly distributed ran- 

a fuzzy image is obtained. Although in these examples, "oise between +5 and — 5 db to the decoded im- 

N4=l and is thus an identity filter, other values for N4 age- Thus, after the nonedge regions are filtered, an 

can be used to alter the enhanced image. additional pass may be made over the nonedge regions 

As shown in FIG. 8, a typical vector quantization in order to add the random noise, 
image transmitting system 96 includes a transmitter 98, 30 The present invention is useful in designing high 
a transmission channel 100, and a receiver 102. In a quality image transmitting systems with existing analog 
vector quantizer transmitting system, the transmitter telephone networks or digital networks like DDS, 
includes a vector quantizer codebook and encoder; the ISDN or PSDN. An immediate application is con- 
receiver includes the same codebook and a decoder. ceived to be in high quality facsimile transmission sys- 
The decoder produces a decoded image from the en- 35 tems. 

coded output- vector representation that is received. The embodiments of the invention in which an cxclu- 

The enhancement system 104 of the present invention sive propeny or privilege is claimed are defmed as 

processes the decoded image {D(m,n)} to produce an foDows- 

enhanced image {E(m^)}. It is to be understood that 1. a method for digital image enhancement using a 

any miage suffenng from the problen^ of staircase ef- 40 processing system including means for receiving ^d 

rv^»5"^''"^^^^ displaying' iiage comprised of decoded ^eTdijS 

ular noise, may be enhanced by enhancement svstem *u^a^*^ JLr.^-^ j j • • 

104. The vector quantization e^e mustxLi o^ ' f*,?? ff^^ edge and nonedge regions m the 

application of the enhancement system. '3'! n>ethod compnsmg the steps of: 

In one actual embodiment, the enhancement system 45 S 1. . 

of the present invention is incoiporated into a SUN differentiatog between edge pixel data and no- 

3/60 workstation including a color monitor. The moni. J^rl^ft^^^ r ^ 

tor provides 1152 X 900 pixel resolution. The color mon. ^"^l ^"^^"^^^ edge pixel by identifymg a first edge 

itor is capable of displaying 256 levels of grey for each ^^^^^ ""^ ^^ion includ- 

pixel. The image that is enhanced may be generated by 50 determining a filtered edge 

a number of sources such as a television camera, an P"^ ^^^^ °° pixel data in said first edge 

image scanner, etc. The image to be enhanced may be a region, such that the staircase effects appcar- 

prestored image, in which case the data describing the filtered edge pixek are displayed are 

image arc stored in memory space accessible to the decreased; 

workstation. Alternatively, the iniage may be enhanced 55 filtering each nonedge pixel by identifying a first 

in real time, i.e., as it is generated and decoded at the nonedge image region, said fist nonedge image 

workstation. region including said nonedge pixel, and determin- 

One benefit of the present enhancement system is that a filtered nonedge pixel value based on pixel 

it can be integrated into an interactive system and the ^ nonedge image region, such that 

edge threshold parameters and the window size param- GO the loss of detail and contouring effects appearing 

eters may be set interactivley. Additionally, an operator when said filtered nonedge pixels are displayed are 

may select multipass filtering and input the parameter decreased; and 

values to be used in each pass. An operator who is (e) combining said filtered edge pixel data and said 

familiar with the operation of the enhancement system filtered nonedge pixel data to form an enhanced 

and who has viewed the decoded image {D(m,n)} may 65 image. 

select and input values for the parameters to obtain the 2. A method as claimed in claim 1, wherein said 

most desirable enhancement based on the content of the method further includes the steps of: rcfiltering each 

decoded image. Using an interactive method, several edge 
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(a) pixel by identifying a second edge image region, 
said second edge image region including said edge 
pixel, and determining a rcfiltcred edge pixel value 
based on pixel data in said second edge image re- 
gion, such that the staircase effects appearing in 
said edge region data when said refiltered edge 
pixels are displayed are further decreased; 

(b) filtering each nonedge pixel by identifying a sec- 
ond nonedge image region, said second nonedge 
image region including said nonedge pixel, and 
determining a rcfiltcred nonedge pixel value based 
on pixel data in said second nonedge image region, 
such that staircase effects appearing in said no- 
nedge region data when said refiltered nonedge 
pixels are displayed are further decreased; and 

(c) combining said refiltered edge region pixel data 
and said refiltered nonedge region pixel data to 
form a second enhanced image. 

3. A method as claimed in claim 1, wherein prior to 
differentiating between the edge and nonedge regions, a 
subimage of said digital image is formed, such that said 
digital image is comprised of said subimage and an unfil- 
tered portion of said digital image, said edge and no- 
nedge data of said subimage are differentiated, filtered, 25 
and combined to form an enhanced subimage, and said 
enhanced image is formed by combining said enhanced 
subimage with said unfiltered portion of said digital 
image. 

4. A method as claimed in claim 1 or 2, wherein said 
step of differentiating between edge and nonedge re- 
gions is carried out by analyzing said digital image via 
an edge detector. 

5. A method as claimed in claim 1, further comprising 
the step of adding random noise to said nonedge data 
after said step of filtering said nonedge data. 

6. A method as claimed in claim 1 or 2, wherein said 
step of determining a filtered edge pixel value based on 
pixel data in an edge image region includes the steps of 40 
ordering said pixel data in said edge image region, and 
selecting a median value from said ordered pixels as said 
filtered edge pixel value. 
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7. A method as claimed in claim 1 or 2, wherein said 
step of determining a filtered nonedge pixel value based 
on said pixel data in a nonedge image region includes 
the steps of approximating a mean value for said pixel 

5 data in said nonedge image region via a statistical D-fil- 
tcring process, and selecting said approximated mean 
value as said filtered nonedge pixel value. 

8. A method as claimed in claim 1 or 2, wherein said 
step of differentiation requires edge identification 

10 threshold parameters, and said edge image regions and 
said nonedge image regions are defined by window 
parameters, and 
said method further includes the step of receiving 
values for said threshold parameters and said win- 
dow parameters prior to the steps of differentiating 
and filtering. 

9. An apparatus for enhancing a digital image, said 
digital image comprised of decoded digital data, the 
decoded digital data defining edge and nonedge regions 
in the image, said apparatus comprising: 

(a) means for receiving a digital image; 

(b) edge detection means for differentiating between 
the edge pixel data and the nonedge pixel data in 
said digital image; 

(c) nonlinear edge filtering means for filtering each 
said edge pixel data by identifying a first edge 
image region including a plurality of pixels such 
that the staircase effects in said filtered edge pixel 
data are decreased; 

(d) nonlinear nonedge filtering means for filtering 
each said nonedge pixel data by identifying a first 
edge image region including a plurality of pixels 
such that the loss of detail and contouring effects in 
said filtered nonedge pixel data is decreased; and 

(e) combination means for combining said filtering 
edge pixel data and said filtered nonedge pixel data 
to produce an enhanced digital image. 

10. An apparatus as claimed in claim 9, wherein said 
nonlinear edge filtering means is a median filter. 

11. An apparatus as claimed in claim 9, wherein said 
nonlinear nonedge filtering means is a statistical D-fU- 
ter. 
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